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DETAILED ACTION 



Background 

1. This Final Office Action is responsive to the following communications: 
Amendment filed on 8/2/2007. 

2. Claims 1-48 are pending in this case. Claims 1, 17, 29, and 41 are in 
independent form. 

Response to Amendment 

3. Applicant's Amendment of the Abstract obviates the Objections made by 
the Examiner in the previous Office Action (Mail dated: 4/6/2007) nwith resect to 
length. The Objection(s) is/are withdrawn. 

4. Arguments (filed on 8/2/2007) concerning the Examiner's Rejections of 
claims 1-48 made under 35 U.S.C. § 102(e) in the previous Office Action (Mail dated: 
4/6/2007) have been fully considered but are not persuasive. The rejections are 
maintained, for the reasons provided hereunder. 

Claim Rejections-35 U.S.C. § 102 

5. The following is a quotation of the appropriate paragraphs of 35 
U.S.C. 102 that form the basis for the rejections under this section made in this Office 
action: 
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(e) the invention was described in (1) an application for patent, 
published under section 122 (b) , by another filed in the United 
States before the invention by the applicant for patent or (2) a 
patent granted on an application for patent by another filed in 
the United States before the invention by the applicant for 
patent, except that an international application filed under the 
treaty defined in section 351(a) shall have the effects for 
purposes of .this subsection of an application filed in the United 
States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in 
the English language. 



6. Claims 1-48 are rejected under 35 U.S.C. 102(e) as being anticipated by 
Edwin A. Suominen (US Pre-Grant Publication 2003/0055655 Al)(hereinafter 
Suominen). 



I. Citation of Prior Art 



A reference to specific paragraphs, columns, pages, or figures in a cited prior art 
reference is not limited to preferred embodiments or any specific examples 1 . It is well 
settled that a prior art reference, in its entirety, must be considered for all that it 
expressly teaches and fairly suggests to one having ordinary skill in the art 2 . Stated 
differently, a prior art disclosure reading on a limitation of Applicant's claim cannot be 
ignored on the ground that other embodiments disclosed were instead cited. Therefore, 
the Examiner's citation to a specific portion of a single prior art reference is not 
intended to exclusively dictate, but rather, to demonstrate an exemplary disclosure 
commensurate with the specific limitations being addressed. 



IL General Discussion of the Applied Prior Art. 



1 In re Heck, 699 F.2d 1331, 1332-33, 216 USPQ 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 
397 F.2d 1006, 1009, 158 USPQ 275, 277 (CCPA 1968). 

2 Upsher-Smith Labs. v. Pamlab, LLC, 412 F.3d 1319, 1323, 75 USPQ2d 1213, 1215 (Fed. Cir. 2005); 
In re Fritch, 972 F.2d 1260, 1264, 23 USPQ2d 1780, 1782 (Fed. Cir. 1992); Merck & Co. v. Biocraft 
Labs., Inc., 874 F.2d 804, 807, 10 USPQ2d 1843, 1846 (Fed. Cir. 1989); In re Fracalossi, 681 F.2d 
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Suominen teaches different multiple types of message recognition using a shared 
language model. Message recognition performed responsive to a first type of message 
input (e.g., speech), to provide text data in accordance with both the shared language 
model and a first model specific to the first type of message recognition (e.g., an acoustic 
model). Message recognition of a second type is performed responsive to a second type 
of message input (e.g., handwriting), to provide text data in accordance with both the 
shared language model and a second model specific to the second type of message 
recognition (e.g., a model that determines basic units of handwriting conveyed by 
freehand input). Accuracy of both such message recognizers can be improved by user 
correction of mistakes in recognition by either one of them. 

III. Prior Art Anticipation of Claimed Limitations. 

As to independent claim 1, Suominen describe(s): A method comprising: 
receiving electronic ink input ("...freehand input to generate and edit text data...," para. 
[0041]); generating a list of machine-generated text candidates based on the electronic 
ink input ("...text that has been generated (erroneously) by a message recognizer...," 
para. [0130]), the list including a first machine-generated text candidate and 
alternative machine-generated text candidates ("...a list of alternative text comparable 
to the selection is provided...," para. [0132]); converting the electronic ink input to the 
first machine-generated text candidate ("...shown by way of example in TABLE IV and 
TABLE V...," para. [0132]); displaying the first machine-generated text candidate 



792, 794 n.l, 215 USPQ 569, 570 n.l (CCPA 1982); In re Lamberti, 545 F.2d 747, 750, 192 USPQ 
278, 280 (CCPA 1976); In re Bozek, 416 F.2d 1385, 1390, 163 USPQ 545, 549 (CCPA 1969). 
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("...displayed list of alternative text ...," para. [0133]); receiving speech input ("...voice 
input...," para. [0093]); converting the speech input to second machine-generated text 
("...Speech recognizer 162 generates text data responsive to voice input from user 240 
and provides the text data to editor 168...," para. [0045]), wherein the second machine- 
generated text is one of the alternative machine-generated text candidates and the list 
of machine-generated text candidates functions as a dictionary used for converting the 
speech input 

At step 1430, a list of alternative text comparable to the selection is 
provided. The list includes text segments that each received a high score 
(but not the maximum score) during hypothesis testing of the message 
recognizer. Two lists of alternative text of the type that may be produced 
at step 1430 are shown by way of example in TABLE IV and TABLE V 
below. As discussed below, the lists of TABLE IV and TABLE V were 
displayed in a correction mode of a conventional computer configured as a 
speech recognizer by executing conventional speech recognition software, 
marketed as NATURALLY SPEAKING, Preferred Edition, version 3.01, 
by Dragon Systems, Inc. 

(para. [0132]); and replacing the first machine-generated text candidate with the 
second machine-generated text ("...desired modifications to text data generated by 
speech recognizer 162 in accordance with editing commands communicated by freehand 
input and interpreted by tablet interface 164....," para. [0097]). 

As to dependent claim 2, which depends from claim 1, Suominen further 
disclose(s): The method of claim 1, wherein the first machine-generated text candidate 
is a word ("...a word ...," para. [0004]). 



As to dependent claim 3, which depends from claim 1, Suominen further 
disclose(s): The method of claim 1, wherein the first machine-generated text candidate 
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is a portion of a word ("...text segments are referred to herein as "stop segments," and 
may include more or less text than a single word...," para. [0148]). 

As to dependent claim 4, which depends from claim 1, Suominen further 
disclose(s): The method according to claim 1, further comprising receiving input 
selecting the first machine-generated text candidate prior to receiving the speech input 
("...the selected text segment is modified responsive to the character input....," para. 
[0134]). . 

As to dependent claim 5, which depends from claim 4, Suominen further 
disclose(s): The method according to claim 4, wherein the selecting includes touching a 
user input device to a digitizer screen at a location corresponding to the first machine- 
generated text candidate ("...by stylus tap somewhere on the text...," para. [0092]). 

As to dependent claim 6, which depends from claim 4, Suominen further 
disclose(s): The method according to claim 4, wherein the first machine-generated text 
candidate is a group of words or part of a word ("...a phrase of several words...," para. 
[0131]). 

As to dependent claim 7, which depends from claim 1, Suominen further 
disclose(s): The method according to claim 1, further including displaying the list of 
machine-generated text candidates prior to receiving the speech input ("...the list of 
alternative text is updated, based on character data communicated after initiation of 
the correction mode...," para. [0133]). 
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As to dependent claim 8, which depends from claim 7, Suominen further 
disclose(s): The method according to claim 7, wherein said step of displaying the 
alternative machine-generated text candidates further includes displaying the 
alternative machine-generated text candidates in the list in an order based on a 
confidence level that each alternative machine- generated text candidate corresponds to 
the electronic ink input ("...The list includes text segments that each received a high 
score (but not the maximum score) during hypothesis testing of the message recognizer. 
Two lists of alternative text of the type that may be produced...," para. [0132]). 

As to dependent claim 9, which depends from claim 1, Suominen further 
disclose(s): The method according to claim 1, wherein the alternative machine- 
generated text candidates include machine-generated text candidates based on the 
electronic ink input generated by a handwriting recognition engine ("...handwriting 
recognition engine...," para. [0042]). 

As to dependent claim 10, which depends from claim 9, Suominen further 
disclose(s): The method according to claim 9, wherein the alternative machine- 
generated text candidates include machine-generated text candidates based on the 
electronic ink input generated in accordance with a statistical language model 
("...Probability Pr(w.sub.iw.sub.j.vertline.V) may be computed (at least as an estimate) 
by any suitable statistical technique. For example, probability 
Pr(w.sub.iw.sub.j.vertline.V) may be computed for a given combination of text segments 
w.sub.i and w.sub.j based on the total number of occurrence of segment wi in matrix Z, 
divided by the total number of occurrences of all segments in matrix Z. The probability 
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of shared cluster occurrence of text segments w.sub.i and w.sub.j may be estimated as 
the ratio between (1) the number of times text segments w.sub.i and w.sub.j can be 
paired together within semantic clusters of the corpus and (2) the total number of text 
segments in the corpus....," para. [0173]). 

As to dependent claim 11, which depends from claim 10, Suominen further 
disclose(s): The method according to claim 10, further comprising displaying the 
machine-generated text candidates generated by the handwriting recognition engine 
and subsequently displaying the machine-generated text candidates generated in 
accordance with the statistical language model ("...Language model 350 includes a 
syntactic model 352 and a semantic model 354. Syntactic model 352 provides a set of a 
priori probabilities based on a local word context, and may be a conventional N-gram 
model. Semantic model 354 provides probabilities based on semantic relationships 
without regard to the particular syntax used to express those semantic 
relationships....," para. [0146]). 

As to dependent claim 12, which depends from claim 11, Suominen further 
disclose(s): The method according to claim 11, further comprising receiving input 
requesting the display of the machine- generated text candidates generated in 
accordance with the statistical language model while displaying the machine-generated 
text candidates generated by the handwriting recognition engine ("...Handwriting 
recognizer 314 then derives units of language (again, strings of text data) from the 
units of handwriting in accordance with language model 350....," para. [0145]). 
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As to dependent claim 13, which depends from claim 1, Suominen further 
disclose(s): The method according to claim 1, wherein the alternative machine- 
generated text candidates include text candidates based on the electronic ink input 
generated by a statistical language model ("...Probability Pr(w.sub.iw.sub.j.vertline.V) 
may be computed (at least as an estimate) by any suitable statistical technique. For 
example, probability Pr(w.sub.iw.sub.j.vertline.V) may be computed for a given 
combination of text segments w.sub.i and w.sub.j based on the total number of 
occurrence of segment wi in matrix Z, divided by the total number of occurrences of all 
segments in matrix Z. The probability of shared cluster occurrence of text segments 
w.sub.i and w.sub.j may be estimated as the ratio between (1) the number of times text 
segments w.sub.i and w.sub.j can be paired together within semantic clusters of the 
corpus and (2) the total number of text segments in the corpus....," para. [0173]). 

As to dependent claim 14, which depends from claim 1, Suominen further 
disclose(s): The method according to claim 1, wherein the step of converting the speech 
input to the second machine-generated text includes determining if the speech input 
corresponds to one of the alternative machine-generated text candidates ("...Speech 
recognizer 312 generates text data in accordance with acoustic model 356 and shared 
language model 350....," para. [0126])(emphasis added); and converting the speech input 
to the corresponding alternative machine-generated text candidate when the speech 
input corresponds to the alternative machine-generated text candidate ("...speech 
recognizer such as a computer executing the NATURALLY SPEAKING software, the 
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modifications to the selected text segment are entered when the user selects an "OK" 
control of a displayed user interface dialog..../' para. [0134]). 

As to dependent claim 15, which depends from claim 1, Suominen further 
disclose(s): The method according to claim 1, wherein further comprising the step of 
receiving an input confirming that the second machine-generated text candidate should 
replace the first machine-generated text candidate prior to performing said step of 
replacing (when the user selects an "OK" control of a displayed user interface dialog....," 
para. [0134]). 

As to dependent claim 16, this claim differs from claim 1 only in that it is directed 
to a product defined by the process of claim 1. Accordingly, this claim is rejected for the 
same reasons set forth in the treatment of claim 1, above. 

As to claims 17-28, these claims differ from claims 1-16, respectively, only in 
that they are directed to objects whereas claims 1-16 are directed to text. Since "text" is 
a type of object ("The machine-generated objects may correspond to words, lines, and/or 
other groupings of machine-generated text.", see applicants disclosure at para. [27]), 
claims 17-28, are rejected for the same reasons set forth in the treatment of claims 1- 
16, respectively. 

As to claims 29-40, these system claims differ from claims 1-15 in that they are 
directed to a system for carrying out the methods defined by claims 1-15. Accordingly, 
claims 29-40 are rejected for the same reasons set forth in the treatment of claims 1-15 
above. 
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As to claims 41-48, these system claims differ from claims 29, 31, 34, and 36-40 
in that they are directed to "objects" whereas claims 29, 31, 34, and 36-40 are directed 
to "text." Since "text" is a type of object ("The machine-generated objects may 
correspond to words, lines, and/or other groupings of machine-generated text.", see 
applicants disclosure at para. [27]), therefore, claims 41-48, are rejected for the same 
reasons set forth in the treatment of claims 29, 31, 34, and 36-40. 

Response to Arguments 

7. Applicant arguments, see pp. 12 filed 8/2/2007, with respect to the 35 
U.S.C. §102(e) Rejections cited by the Examiner in the previous Office Action (Mail 
dated: 4/6/2007), have been fully considered but are not persuasive. Therefore, the 
rejection(s) have been maintained. 

Applicant Argues, 

According to the features of the present invention, electronic ink input 
is received and a list of machine-generated text candidates is created 
based on the electronic ink input, said list including a first machine- 
generated text candidate and alternative machine-generated text 
candidates. Speech input is then received. This speech input is 
converted to second machine- generated text, wherein the second 
machine-generated text is one of the alternative machine- 
generated text candidates and the list of machine-generated 
text candidates functions as a dictionary used for converting 
the speech input. Applicants submit that Suominen fails to disclose 
or suggest at least this feature of independent claim 1. 

(emphasis in the original) (Applicant REMARKS, filed 8/2/2007, Page 12). 

In response, the examiner points to the following teaching: 
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[0132] At step 1430, a list of alternative text comparable to the 
selection is provided. The list includes text segments that each received 
a high score (but not the maximum score) during hypothesis testing of the 
message recognizer. Two lists of alternative text of the type that may be 
produced at step 1430 are shown by way of example in TABLE IV and 
TABLE V below. As discussed below, the lists of TABLE IV and TABLE V 
were displayed in a correction mode of a conventional computer 
configured as a speech recognizer by executing conventional speech 
recognition software, marketed as NATURALLY SPEAKING, Preferred 
Edition, version 3.01, by Dragon Systems, Inc. 

[0133] At step 1440, the list of alternative text is updated, based on 
character data communicated after initiation of the correction mode. In a 
conventional speech recognizer such as a computer executing the 
NATURALLY SPEAKING software, the displayed list of 
alternative text changes as different characters are entered by a 
user using character input. 

Applicant Argues, 

The paragraph relied upon by the Examiner, [0172], discusses the 
computation of the probability of shared cluster occurrence. Calculating 
the probability of shared cluster occurrence is not the same as the 
features of the present invention whereby a list of text candidates based 
on a first input is used as a dictionary basis for converting a second input 
to machine- generated text, wherein the second input is converted to one of 
the text candidates in the list. 

(Applicant REMARKS, page 13.). 

In response, the examiner points to the following teaching: 

A user context is the context of all documents generated by a particular 
user. A user context may also be viewed as a context of all documents 
generated by a group of users, such as members of a particular 
collaborative group. Examples of collaborative groups include project 
teams, institutions, corporations, and departments. Members of a 
collaborative group may share a common pool of stop segments. A stop 
segment designated by any member of the group as being outside his or 
her user context is then considered to be outside the user context of each 
group member. In one variation, a leader of the group may review a list of 
pooled stop segments and edit the list. In another variation, each group 
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member may remove pooled user stop segments (at least from operation of 
message recognition responsive to his or her message input) that the 
member considers to be inside his or her particular user context. 

(para. [0156]) 

Applicant Argues, 

Finally, Suominen does not disclose converting a speech input to machine- 
generated text based on the alleged text segments chosen from the corpus. 
In the present invention, the speech input is converted to be one of the 
alterative machine-generated text candidates, and the list of machine- 
generated text candidates generated based on the electronic ink input 
functions as a dictionary used for converting the speech input. 

(Applicant REMARKS, page 13). 

However, Suominen teaches that the list of machine-generated text candidates 

are generated based on electronic ink input, that is functioning as a dictionary for 

converting speech input: 

At step 1440, the list of alternative text is updated, based on character 
data communicated after initiation of the correction mode. In a 
conventional speech recognizer such as a computer executing the 
NATURALLY SPEAKING software, the displayed list of alternative text 
changes as different characters are entered by a user using character 
input. 

(para. [0133]). 

Conclusion 

8. Although not relied upon, the following prior art is made of record because 
it considered pertinent to applicant's disclosure: 

Suominen, Edwin A. US 20030055655 A1 Text processing system 

Bangalore, Srinivas et al. US 200401 19754 A1 Context-sensitive interface widgets for multi-modal dialog 

systems 

Bangalore, Srinivas et al. US 20040122674 A1 Context-sensitive interface widgets for multi-modal dialog 

systems 



I 
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Suominen, Edwin A. US 20050171783 A1 Message recognition using shared language model 
Bushey, Robert R. et al. US 20050183032 A1 System and method for managing recognition errors in a 

multiple dialog state environment 
Ditzik; Richard Joseph US 6167376 A Computer system with integrated telephony, handwriting and 

speech recognition functions 



Conclusion 

9. All prior art made of record in this Office Action or as cited on form PTO- 
892 notwithstanding being relied upon, is considered pertinent to applicant's disclosure. 
Therefore, Applicant is required under 37 CFR §1.1 11(c) to consider these references 
fully when responding to this Office Action. 

THIS ACTION IS MADE FINAL, Applicant is reminded of the extension of 
time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire 
THREE MONTHS from the mailing date of this action. In the event a first reply is 
filed within TWO MONTHS of the mailing date of this final action and the advisory 
action is not mailed until after the end of the THREE-MONTH shortened statutory 
period, then the shortened statutory period will expire on the date the advisory 
action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be 
calculated from the mailing date of the advisory action. In no event, however, will 
the statutory period for reply expire later than SIX MONTHS from the mailing date 
of this final action. 



I 
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10. Any inquiry concerning this communication or earlier communications 
from the Examiner should be directed to Samir Termanini at telephone number is (571) 
270-1047. The Examiner can normally be reached from 9 A.M. to 6 P.M., Monday 
through Friday. 

If attempts to reach the Examiner by telephone are unsuccessful, the Examiner's 
supervisor, Stephen S. Hong can be reached on (571) 272-4124. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained, from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR 
only. For more information about the PAIR system, see http://pair-direct.uspto.gov. 
Should you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 





Samir Termanini 
Patent Examiner 
Art Unit 2178 



STEPHEN HONG 
SUPERVISORY PATENT EXAMINER 



